A regularized variance-reduced modified extragradient method for stochastic hierarchical games

The theory of learning in games has so far focused mainly on games with simultaneous moves. Recently, researchers in machine learning have started investigating learning dynamics in games involving hierarchical decision-making. We consider an $N$-player hierarchical game in which the $i$th player's objective comprises of an expectation-valued term, parametrized by rival decisions, and a hierarchical term. Such a framework allows for capturing a broad range of stochastic hierarchical optimization problems, Stackelberg equilibrium problems, and leader-follower games. We develop an iteratively regularized and smoothed variance-reduced modified extragradient framework for learning hierarchical equilibria in a stochastic setting. We equip our analysis with rate statements, complexity guarantees, and almost-sure convergence claims. We then extend these statements to settings where the lower-level problem is solved inexactly and provide the corresponding rate and complexity statements

A relaxed-inertial forward-backward-forward algorithm for Stochastic Generalized Nash equilibrium seeking

In this paper we propose a new operator splitting algorithm for distributed Nash equilibrium seeking under stochastic uncertainty, featuring relaxation and inertial effects. Our work is inspired by recent deterministic operator splitting methods, designed for solving structured monotone inclusion problems. The algorithm is derived from a forward-backward-forward scheme for solving structured monotone inclusion problems featuring a Lipschitz continuous and monotone game operator. To the best of our knowledge, this is the first distributed (generalized) Nash equilibrium seeking algorithm featuring acceleration techniques in stochastic Nash games without assuming cocoercivity. Numerical examples illustrate the effect of inertia and relaxation on the performance of our proposed algorithm

A Novel Denoising Method for Retaining Data Characteristics Brought from Washing Aeroengines

Airlines evaluate the energy-saving and emission reduction effect of washing aeroengines by analyzing the exhaust gas temperature margin (EGTM) data of aeroengines so as to formulate a reasonable washing schedule. The noise in EGTM data must be reduced because they interfere with the analysis. EGTM data will show several step changes after cleaning the aeroengine. These step changes increase the difficulty of denoising because they will be smoothed in the denoising. A denoising method for aeroengine data based on a hybrid model is proposed to meet the needs of accurately evaluating the washing effect. Specifically, the aeroengine data is first decomposed into several components by time and frequency. The amplitude of the component containing the most noise is amplified, and Gaussian noise is added to generate noise-amplified data. Second, a Gated Recurrent Unit Autoencoder (GAE) model is proposed to capture engine data features. The GAE is trained to reconstruct the original data from the amplified noise data to develop its noise reduction ability. The experimental results show that, compared with the current popular algorithms, the proposed denoising method can achieve a better denoising effect, retaining the key characteristics of the aeroengine data

Adaptive Levenberg–Marquardt Algorithm: A New Optimization Strategy for Levenberg–Marquardt Neural Networks

Engineering data are often highly nonlinear and contain high-frequency noise, so the Levenberg–Marquardt (LM) algorithm may not converge when a neural network optimized by the algorithm is trained with engineering data. In this work, we analyzed the reasons for the LM neural network’s poor convergence commonly associated with the LM algorithm. Specifically, the effects of different activation functions such as Sigmoid, Tanh, Rectified Linear Unit (RELU) and Parametric Rectified Linear Unit (PRLU) were evaluated on the general performance of LM neural networks, and special values of LM neural network parameters were found that could make the LM algorithm converge poorly. We proposed an adaptive LM (AdaLM) algorithm to solve the problem of the LM algorithm. The algorithm coordinates the descent direction and the descent step by the iteration number, which can prevent falling into the local minimum value and avoid the influence of the parameter state of LM neural networks. We compared the AdaLM algorithm with the traditional LM algorithm and its variants in terms of accuracy and speed in the context of testing common datasets and aero-engine data, and the results verified the effectiveness of the AdaLM algorithm

Advances on Penetrating Imaging of Building Layout Technique Using Low Frequency Radio Waves

Obtaining the internal layout of an unfamiliar building before entering the building has important practical significance and research value, as it can be applied for various services, such as anti-terrorism operations and disaster relief. Low-frequency electromagnetic waves can propagate through common building materials, and then the target information behind the wall is obtained safely and stably. Therefore, using low frequency radio waves to obtain the information behind the wall has become the research focus in the field of building layout reconstruction. To reveal the development context of this field and predict the possible future development trends, this paper summarizes the domestic and foreign public literature in this field since the onset of the 21st century. The results of the relevant literature indicate that the techniques of using low-frequency electromagnetic waves to reconstruct building layout currently include three types: through-the-wall radar imaging technology based on reflected wave measurement, radio-frequency tomography technology based on transmitted wave measurement, and wall position estimation technology based on multipath signals. These three technologies have achieved several practical research results. This article clarifies the development history of the main content covered by these technologies, which mainly includes the principle of through-the-wall radar imaging of stationary targets behind the wall, the observation mode of building internal structure based on through-the-wall radar, the reconstruction technology of building internal structure on the basis of through-the-wall radar imaging, the inversion technology of building internal structure on the basis of radio-frequency tomography, and the wall position estimation technology based on multipath signals. We also discuss the development trend of this field. In the past two decades, the development history of building layout penetrating imaging using low-frequency radio waves shows a change from the traditional airborne and vehicle-mounted building-layout-reconstruction platforms to new platforms such as microrobots and unmanned aerial vehicles. The corresponding reconstruction method has been developed from the traditional radar imaging technology to a variety of new methods, including image enhancement and sparse reconstruction. The results indicate that the building-layout-reconstruction technology is developing in the direction of systematization, diversification, and intelligence

Stochastic relaxed inertial forward-backward-forward splitting for monotone inclusions in Hilbert spaces

We consider monotone inclusions defined on a Hilbert space where the operator is given by the sum of a maximal monotone operator T and a single-valued monotone, Lipschitz continuous, and expectation-valued operator V. We draw motivation from the seminal work by Attouch and Cabot (Attouch in AMO 80:547–598, 2019, Attouch in MP 184: 243–287) on relaxed inertial methods for monotone inclusions and present a stochastic extension of the relaxed inertial forward–backward-forward method. Facilitated by an online variance reduction strategy via a mini-batch approach, we show that our method produces a sequence that weakly converges to the solution set. Moreover, it is possible to estimate the rate at which the discrete velocity of the stochastic process vanishes. Under strong monotonicity, we demonstrate strong convergence, and give a detailed assessment of the iteration and oracle complexity of the scheme. When the mini-batch is raised at a geometric (polynomial) rate, the rate statement can be strengthened to a linear (suitable polynomial) rate while the oracle complexity of computing an ϵ-solution improves to O(1/ϵ). Importantly, the latter claim allows for possibly biased oracles, a key theoretical advancement allowing for far broader applicability. By defining a restricted gap function based on the Fitzpatrick function, we prove that the expected gap of an averaged sequence diminishes at a sublinear rate of O(1/k) while the oracle complexity of computing a suitably defined ϵ-solution is O(1/ϵ1+a) where a>1. Numerical results on two-stage games and an overlapping group Lasso problem illustrate the advantages of our method compared to competitors